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I. 



INTRODDCTICN 



When is a program considered to be trivial? One answer 
to this question heard very often is ’’When it contains no 
bugs". although this statement might be questionable, the 
converse is true, as there are few nontrivial programs that 
do not certain bugs. As the author of a critical and 
fundamental study of program design states; "...These tugs 
can never be completely exorcised in any program over some 
critical decree of complexity. Six months or even seven 
years after 'final debugging' errors crop up inevitably in 
the best of prog rajas 4 ]. This is a fact one has to live 
with, and there are .only two things one can do about it; 
First to reduce the possibilities for bugs by careful design 
and use cf modern programming techniques, second to devise 
careful testing techniques to detect and locate the hugs 
still remaining in tie program. 

Fig. 1 shows the relationship between hardware and 
software cost in the D.S. during the period from 1955 to 
19£5. Cue to the fact that the software cost continues to 
rise and that about 50S of this cost is for testing and 
integration cf a system [7], it is important to obtain a 
realistic assessment of how much effort has to be spent to 
test the newly designed program based on its sixe, structure 
and characteristics. If one is able to determine ir the 
design stage the best possible structure with respect to the 
error detection capabilities, then bugs can be avoided and 
testing will be reduced. Also early in the development cf a 
project a realistic allocation of coding and testing 
resources could be made. 
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Ficure 1 - SOFTtfABE COST TREND IN THE D,S. 

[ Datamation, Sept. 1974, pg. 75] 

In order to address these problems, a Software Error 
Detection Simulation Model has been developed [7,10]. This 
model was was used to identify program complexity measures 
which were correlated with error detection. Naval Tactical 
Data System picgrams were used for this purpose. 

The structures of these NTDS-programs have been analyzed 
(see Chapter ^I) and put into the form of directed graphs. 

The date gained from the directed graph representation 
were used as inputs for the Error Detection Simulation 
Model. The results gained and the conclusions and 

recommendations draJin from these results are shown in 
Chapter 1/II . For reasons of security, the programs or the 
parts of them are not identified by names. Instead, a 
sequential cumber scheme for identifying the programs has 
been employed. 
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This wcrk is part of a research effort sponsored ty the 
NACC to get software evaluation aids which provide an 
economical assessment of the design and testing effort 
needed for the development of avionics and other complex 
software projects. 



Because it is 
debugging can be 
technigues in the 
chapter shows the 
to the prchlem of 



felt that efforts in testing and in 
more successful if cne employs icdern 
production of programs, an introductory 
relevance of modern programming techniques 
program testing and maintenance. 
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II . DEFINITIONS 



There was originally a lack of commonly used definitions 
for program testing. Only recently has a '’definitional 
framework" eaerged and very good program testing definitions 
are found in Eef. 8, pg. 7 - 14. In order to be consistent 
and to specify the meaning of keywords within this thesis, 
the fcllcwinc definitions have been adopted; 



1 . Etc^raj Struc tur e 



The structure of a program is a description of the 
underly,ing logic and data flow as represented in the 
form cf a directed graph with its set of nodes and 
edges (arcs) . 



2 • Eeacbability Index 



Beackatility index is 
possibilities to get to a 
over all nodes of the 
computed with the formula 



a measurement 
specified node, 
directed graph. 



o f the 
computed 
It is 



R 




to node 



(i) • 



Debugging 



Debugging is the action one takes to locate and 
correct a known or detected error in a program. 
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4 . 



lestic^ 

Testing is the action to check whether a picgram 
meets its specifications and to establish the 
presence of errors in it. 

5 . li f € c ycle of a prog ram 

The life cycle of a program consists of the 
fcllcwing phases: 

- design 

- Coding 

- Cebucging 

- Testing 

- Eroduction and maintenance. 



III. MO^EN PBOGEAM^NG TECHNIQUES 



Iwo recent developments in the theory and practice of 
software cevelcpment are addressed here as important because 
they are relevant not only for the actual writing cf the 
code of the program, but also to debugging, testing, and 
integrating software systems as well, namely the advent of 
modular and structured programming. The advantages of these 
technigues are obvious for the programmer when he develops 
his program, programs written using these technigues are 
easier to read and to understand as far as the flow cf the 
logic is concerned. Also, the tester can better understand 
the logic cf a program when these technigues are employed, 
furthermore, it has been proposed for structured programs to 
eliminate flowcharts as media cf communication [13], sc it 
is necessary tc undexstand how much testing, integration and 
maintenance cf software are influenced by this development. 



A. HCDD12B ifCGBAKMING 



Modular programming is a system to develop programs as a 
set of interrelated individual units (called modules) which 
later can be linked together tc form a complete program [9]* 
Thus modular programming is not simply splitting up a 
program intc several parts (subroutines) , but rather 
dividing the software according to the functions tc be 
performed. The designer faces the one crucial problem which 
will determine success or failure, namely to specify 
completely and carefully the interface between the 
individual mcdules. 
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Modules as individual program units should hav€ the 
following properties: 

(1) Cre ircdule should perfcrir only one basic function 

(2) The size of a module should be such that it is 
easily understood and contains crly a moderate 
ancurt of code 

(3) module should be designed in such a way that it 
has only a few control or data paths 

(4) Cne nodule should process only a small amount of 
data . 



The design of programs in this way leads not only to 
cleaner a r.d more productive coding but also to easier and 
more flexible testing. The advantages with respect to 
debugging ard testing show up in several ways. Single 
modules can be debugged and tested independently from the 
other modules or the main (driver) program. Furthermore, if 
the modules are snail enough, extensive testing generally 
assumed as impossible with the exception of very trivial 
programs, can become manageable. This in turn leads to more 
reliable programs. If all modules of a software project can 
be tested extensively, a highly reliable program can be 
produced. iven if one falls short of this goal - and this 
happens ir most cases due to the very large number of 
possible inputs and program paths - the final program will 
be more reliable and more thoroughly tested than a 
ncn^fflcdular program. The possibility of testing modules 
individually provides for better (more eccncnical) 
allocation of testing resources, because cne does not have 
to wait until the whole program has been completed. However, 
to test individual modules, special test-routines are needed 
as drivers and if other modules must interact, dummy modules 
must be created if tie real modules are not yet available or 
not yet tested. 
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One final point in favour of modular programming has to 
be made: Normally, no production program is completed unril 
the day when it is no longer used, i.e. every running 
production program has to be maintained and adapted to new 
considerations and situations. Because of the simplicity of 
the overall crgani 2 ation of modular programs this software 
maintenance is alleviated since interactions between modules 
are more easily understood; hence, the effect of program 
changes is easier to identify. Also only the modules 
affected ky the change have to be tested (together with the 
main program and interacting modules) . 



fl. SIBDClDfiir PECGEAKMING 



Having coded a program in the above described 
modularized fashion; there is still room for improvement. 
Since Dijkstra’s famous letter to the editor of the 
Communicaticrs of the ACM in which he proposed to eliminate 
GO-IO statements [5]^ the concept of Structured Programming 
has evolved and led to further simplification of the coding 
process. 

Simplification means here not that the actual code is 
easier to write - although this might be the case toe for a 
programmer who is familiar with the concept and can think in 
these terms - but the code produced and the control secuence 
of the finished program is simpler than in a non structured 
pregram. Ibis simplification has been theoretically 
demonstrated ty Boehmi and Jacepini as early as 1966 [3]. 
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Although there are as many interpretations of what 
Structured Erogramming is as there are authors oc this 
topic, tte following features are essential and commcn to 
this concept: 

(1) ICP-^ICWN Desggn, i.e. the design starts at a very 
general level and proceeds stepwise to the specific 
and detailed tasks 

(2) Modular Design 

(3) limited possibilities to control the logic flow of 
the program, namely only 

* seguential 

♦ conditional: IF - THEN - ELSE 

=* iterative: DO - WHILE 

statements are allowed. 

Whereas the so called block-structured languages like 
ALGOL cr EL/I lend themselves to this form of coding 
(although GC-IO statements are provided' by the language), 
even in ECBTEAN the implementation of seme of the basic 
principles of Structured Programming is possible if the 
programmer concerned with a structural flew cf his program 
cheeses the branching caused by unavoidable GC-TO 
statements carefully. 

Eaker [1] shows that the application of Structured 
Programming combined with the ’’Chief Programmer Team Method" 
of organising a software project [2] can bring measurable 
improvements in software development, in the coding as well 
as in the debugging and in the testing stage. Due to the 
fact, that Structured Programming implies Modular Programming 
the same advantages hold here too, i.e. the software is 
easier tc test and tp maintain after release. 
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IV. 



IHJ PROB LEM CF P BOGB AM COMBIE XIT Y 



The inpact of th£ programniing technigues descrihed afcove 
cn the €CCECiric development of reliable and maintainable 
software is directly related to the problem of program 
complexity. There is so far no generally adopted definition 
of what program complexity really means. The definition is 
dependent or the context in which one wants to examine 
program complexity. Here complexity is defined as structural 
properties of a program that affect the ability to detect 
errors. 

Dnder the condition that the structure of a program is 
described by a directed graph, the following criteria can be 
used to measure its complexity: 

1. Number of nodes 

2. Number of arcs 

3. Number of possible paths through the program 

4. Number of source statements 

5. Averace path length (source statements per path, arcs 
per paths) 

6. Beachability index 

7. Fullness index (ratio of actual to maximum number of 
arcs) . 

Although Mills in his contribution to Eef. 8 generates the 
idea of equating program complexity with the difficulty of 
understanding a prpgram and justifies this approach with 
'‘.-■..'the frustration of concocting and demolishing more 
simple minded direct ideas, such as counts of branches, data 
references, etc.", his approach does not help to get a real 
measurement of complexity such that one is able to make a 
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quantitative stateae'Ut how complex a program is. It seems 
that the important point is to relate program complexity to 
the problem area one pursues. The analysis of NT DS-E cgrams 
has given insight in methods to measure complexity vith 
respect to problems of program design and testing. 
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V. 



JBICR detection simulation mod^ 



A. GENEEAL 



A Software Error Detection Simulation Model was 
criginallj developed by T.F. Green in his M.S. Thesis [7] 
and subsequently mpdified by professor G.T. Howard of the 
Naval Postgraduate School. Written in FORTRAN it was 
designed to run on the IBM 360/67 computer of the Naval 
Postgraduate School. Originally it had been tested against 
hypothetical and actual programs. It was shown that 
sinulaticD of error detection was feasible and that 
information cculd be obtained on the relationship between 
error detection and program complexity. However, it was 
necessary to perform additional model feasibility tests by 
using the model on a large number of actual programs. In 
the process of testing some of the original features had to 
be removed and provisions had to be made for cases of 
program behaviour which were unexpected at the time of the 
simulation program design. A detailed description of the 
model with its specific assumptions and capabilities is 
found in Eef. 10, pg,- IV-5 - IV-39. 

E. PRCGBAM EEPRESENTATION 



The prerequisite for the use of the simulation mccel is 
to get the structure of a program that has to be tested in 
the form of a directed graph. A directed graph is a 
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convenient D-eacs to show the structure of prcgrams. It is 
suitable for showing the control flow in a program, measures 
of compleaitj can be derived from this kind of 
representation. In addition, the *’ccntrcl flow graph" as 
this ccmpcsiticn cf structures is sometimes called, is also 
very useful for determining the execution time of a 
structure cn a machine. This representation of pregram 
structures also simplifies the representation of large and 
ccnplex programs because these programs can be broken up in 
logical segirents (modules, procedures, subroutines etc.), 
and the segments can be tested separately from the ether 
parts cf tie program. 



C. CUBRiKT STATDS 0«F THE SIMUIATICN PEOGB^H 



1. 


Input 


Variables 










The 


following input 


variables have to 


be used for 


the si 


mu laticn : 








a . 


MINPUI 


designates the 


number of inputs 


withic each 






replication. 








b. 


NOaODT 


is the number 


of 


replications 


(number of 






paths)^ within every 


repetiticE . 




c . 


NEEPEI 


is the number 


cf 


reseedings 


with errors 






(repetitions) . 








d. 


MEANLE 


designates the 


mean arc length if the arc 






lengths are selected 


at random by 


the pregram 






and are not read 


in . 






. e. 


MEANEE 


designates the mean 


number cf 


instructions 






between errors. 








f . 


N 


is the number cf 


nodes within the 


strucrure. 


g- 


Input 


fer the Adjacency 


Matrix is done in 


a shorthand 
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nctaticn: 

Per €very node with the exception of the 
there is one data card which contains i 
atcut this node in the following 
Identification of the node, number cf arcs 
frcffi this node, identification numbers of 
tc which the arcs go. 

h. Input for the matrix of arc lengths 
siailar to that for the adjacency matrix: 
only as the identifiers for receiving node 
(identifier, number of statements on this a 
be provided. 

i. Input tc plant errors in arcs instead cf le 
pregram seed them at random: Input as for 
arc length, but the number of errors on thi 
tc he specified instead of the number cf st 

j. MCDT specifies the desired output: 

0 = Summary output 

1 = Extensive output (NUMOOT * NHEPEI < 25) 



2- Incut Eorm ats 



Ihe input formats are as follows; 

First data card: (615) aiNPGT, NUMODT, NHBPE 

MEANES, K. 

Secend and following cards; adjacency matrix 
followed hy delimiter-card: S9 in columns 4 and 5. 

Input cards for matrix of arc length (cptio 
7(I5,F5.C); followed by delimiter-card: 99 in col 
5.. 

Input to seed errors manually (optional) : 1615; 

delimiter-card: 99 in columns 4 and 5. 



last nodes 
nf or ration 
seguence: 
emanating 
the nodes 

(opt ional) 
Instead , 
s the pair 
rc) has to 

ttinc the 
matrix of 
s arc has 
atements . 



T, EEAJIIN, 

, (1615); 

n al) : 2l5 , 
umns 4 and 
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Last data caid (output specification): 15. 

Note that all delimiter cards are not optional. 

lim ita tion s 

Ihis simulation program is currently restricted to 
accomodate a maximum number of 30 nodes. The execution time 
for simpler structures (about 10 - 15 ncdes) is within a 
five minute time limit. Larger and more complex structures 
with more nodes and possible paths through the structure 
reguire a 30 minute time frame for the execution of one 
simulated input in 100 replications and 100 repetitions. 

An extension of the limits of the program to 
accomodate larger structures seems to be impractical because 
of the fast rise of memory space and execution time 
reguired . 

ilPSlU Listing 

A listing of the current error detection simulation 
program as it was used for the analysis of the MTDS-prcgrams 
is found in Appendix A. 
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VI. A NALY SIS OF NTDS FROG FANS 



A. GENEFAl 

In order to dempnstrate the practicality of program 
analysis using the Error Detection Simulation Model, Naval 
Tactical Data Systems Programs have been analyzed by 

1. describing the structure by converting the programs 
into the form of directed graphs 

2. running these structures on the error detection 

simulation model and 

3. evaluating the simulation results with respect to 
measures of program complexity. 



B. DESIGN Cl NTDS EflCGEAMS 



^cdular Design 

Ihe design of NTDS programs is characterized by 
Modular Frccramuing, both in general and in detail, and the 
modular design is a characteristic of the hardware as well. 
Also the actual implementation of every NTDS installation 
consists of hardware and software building blocks that are 
composed to fit exactly the need of each installation. 

Although NTDS programs are really programmed in a 
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mcduXar fashicn, tJie term ''module” does net have ths same 
meaning as usual. Mpdule usually refers to basic building 
blocks that are farts of the program, whereas NTDS programs 
are composed of subsystems. The NTDS-”Modules” in turn are 
divided up in parts which correspond to the 
”module”-def initicn of Modular Programming. In NTDS 
terminology these parts are called procedures. NTDS modules 
perform complex tasks such as tracking, display etc. They 
certain a ledium to large number of dependent prccecures. 
These preoedures perform the basic functions intended in 
Modular fregramming such as checking track properties. 
Throughout this discussion, "module” is used as in the NTDS 
system, namely as a complete subsystem for performing 
complex tasks. 

The modular approach is imbedded in a stringent 
hierarchical system which is controlled by the priorities of 
the tasks to be performed. The levels of hierarchy are 
applied to the modules in such a way that only major 
subprograms which are designed to execute distinctive tasks 
can communicate with each other, whereas the procedures 
within the modules can only communicate according to xhe 
level of hierarchy they belong to, with the exception of 
calls to certain system rourines. 

2 . CS-^ Lan guag e 



The NTDS programs are written using the CS- 1 high 
level language compiler [6]. This language has the advantage 
that it is well suited to the application area, namely 
tactical pregrams which run under severe constraints 
regarding time and memory space availability. Tables are 
searched in a very effective way, and another interesting 
feature is that assembly code can be interspersed within the 
high level code of the program. This fact gives the 
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frcgrammer a powerful means for controlling the hardware 
which in tnrc facilitates the production of effective code. 



C. LIBECIED GBAEH CCNSTEUCTICN 



In order to ottain the desired statistics and to analyze 
the data- and control flow of a single NIES-progran , the 
following method has been developed and used: 

1. Cne oomplete module from an existing and currently 
operating KIDS program has been put into the form of 
a directed graph. The module has heen deccapcsed 
intc the procedures it contains, and every procedure 
is treated separately. Due to the modular design 
tbcughcut the program, no logical difficulties arise 
here, because every procedure has only one entrance 
and cne exit point, i.e. the interface for 
interacting procedures within the module is uniquely 
defined. For each procedure the directed graph and 
the adjacency matrix have been constructed. As 
quantitative measurements the number of nodes, arcs, 
paths, loops, source statements, machine 
instructions, source statements per arc, and machine 
instructions per arc have been compiled. 

2. The same work was done for randomly selected 
procedures from one other important module of the 
same prcgran in order to obtain comparative results 
and to relate the reported number of errors to the 
different modules. 



24 



3. Ecr the construction of the directed graphs and the 
gathering of the several statistics the following 
assumptions have been made: 

a. Nodes are associated with 



(.1) Procedure entrance and exits 

(2) IF-statements (decision points) 

(3) Points where paths merge 

(Si) Procedure calls within the module 

(5) Beginning and ending cf loops 

t. All nodes within the module are distinct. 
They have individually assigned numbers (some 
nodes are indicated as "dumity'* nodes) , and 
they are counted only once, namely ir the 
procedure they belong to. 



Entrance and exit nodes of a called procedure 
are regarded as "transient" nodes within the 
calling procedure, and one "transient arc" 
connects both transient nodes. This 
transient arc represents all the arcs inside 
the called procedure. The transient arcs are 
indicated in the drawings by a dashed line. 
Transient nodes have either the number cf the 
entrance node of the called procedure or they 
are denoted by letters tc distinguish them 
frcm the original nodes of the corresponding 
procedure. 



d. The langth of every arc is indicated as the 
number of source statements cr the number of 
machine instructions respectively. Ir the 
analysis the number cf source statements has 
been used because programs are normally 
written in a high level language and this is 
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the point where errors are introduced into 
the program. 

4. Normally, the numbers of both source statements and 
machine instructions have been counted in the arc 
where the statements appear. However, because 

Il-statements and procedure calls result in 

branching, they have been counted in the arc leading 
tc the corresponding node. Whereas for the counting 
of machine instructions, it would be possible in the 
case cf an IF-statement to split the instruction 
sequence according tc the arcs emanating froa the 
decision point, this is not feasible for the source 
statement which contains the elements of both arcs 
eaanating from it; it cannot be split. 

The structures obtained from bcth modules anc the 
compiled statistics are found in Appendix E. The following 
figure shews how tc read the structure diagrams: 




Figure 2 - EXAMPLE OF FROGHAM SIEUCT05F 
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c. 



EBSOB DEIECTION SIMOLATICN ON THESE STBDCTUBES 



The structures which were converted into directed craphs 
for Module One were screened to determine their suitability 
for errcr detection simulation. It would have teen 
desirable to select a random sample of the structures. 
However, it was necessary to choose structures which wculd 
not require excessive amounts of memory space and CPU time 
during the simulation. In addition, the structures were to 
have at least two or more paths. In the case of Module Two 
it was leasitle tp use a random sample because a high 
percentage of the structures fell within the memory space 
and the CPU time limitations of the model. 

The input data for the simulation were taken from the 
actual programs, including the number cf source statements 
for every arc. The recorded number of errors per module was 
used to calculate the mean number of instructions between 
errors, which is used for seeding errors in the simulation 
model. Seeding the errors was done randomly by the 
simulation program. However, it was provided thar no errors 
were seeded at arcs containing zero instructions (control 
arcs) . 

The simulation was run with one input, 100 replications 
and 100 repetitions (reseedings) , and the average number of 
errors found by one input was obtained. although seme of 
the structures were small, and a higher number of 
repetitions and replications could have been run, the same 
simulation parameters were used for each structure in order 
to obtain ccaparable results. 
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E. 



RESUIIS CE THE A^NALYSIS 



Erom th€ average of errors found by one input in each 
procedure the average percentage of errors fcund against the 
errors expected within the procedure was obtained. These 
results were plotted against various complexity measures/ 
e.g. the cumber of paths. Although the results varied 
scmewhat between tie modules, it was possible to establish 
relationships between structural properties and error 
detection capabilities. 

The differences in results between modules can he traced 
to several factors: 

1, Different sample sizes: 

From Module One 32 procedures were used, 16 
procedures were randomly selected frcm Module Two. 

2. The different size of the modules: 

Module One had 97, and Module Two had 155 procedures, 

3. Differences in program design and programming style: 
Module Two was modularized to a much larger extent 
than Module One. It was hard to find a sufficient 
number of paths within randomly selected procedures 
of Module T wo. 

4, Different number of reported errors: 

Although Module Two was 1.6 times larger than Module 
One, it had only about two-thirds the number of 
errors . 

The following diagrams show the percentage of average 
errors fcund against the expected number of errors for the 
structures of both modules. 
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figure 6 - PERCENTAGE ERRORS FOUND VS. SOURCE STATEMENTS 



30 



. . I, A™ W'"'* 








Percent of expected errors Percent of expected errors 



2 



ficdule Two 



100 1 




I i I 1 I I t 

5 10 15 20 25 30 35 

nodes 

figure 7 - PEBCENTAGZ EEBOES FOONE VS. NODES 




31 



Percent of expected errors Percent of expected errors 



100 , 



50 




5 10 15 20 25 30 35 

paths 

Figure 9 - PERCENTAGE ERRORS FODNC VS. PATHS 




Figure 1C 



PERCENTAGE ERRORS 



FOUND VS. 



SOURCE STATEHETS 



32 



I 



The 


curves shown represent 


ex 


pcnential 


approximations 


to the datapoints according 


to 


zhe 


formula 


y=a*€** (-b*x) 


which was found to represent 


the 


rel 


ati enship 


best. A least 


Square fit was used. 









All diagrams show seme relationship between error 
detection and complexity. Module One with its larger sample 
size shews this relationship more than Module Two fer the 
number of paths. This seems logical because a large number 
cf paths reduces the ability to detect errors in a program. 
It appears that the number of paths could be used as a 
measure cf program complexity for design and testing 
purposes . 

In order to rank the approximations, a squared error 
factor has teen computed for every complexity measure as 
follows : 



Error 
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Nodes 
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This computation 


shojjs that 



paths is the best approximat 
method used, incther interesting 
approximated relationship betwee 
and the number cf source stateme 



Factor 
Mod. 2 

4430 

3933 

4666 

1808 



Module 


One 


the 


n 


umb 


er 


of 


ed com 


plexity 


meas 


u 


re 


tv 


t he 


aspec 


t found 


was 




the 




well 


n perc 


entage 


cf er 


r 


ers 


f 


cund 


nt s in 


module 


Two . 











33 



VII. DSE OF THE RESULTS 



A. AILS FCE SOFTWARE DEVELOEflENT 

This methcd cf program analysis provides the so 
manager vith information for selecting structures easi 
the design process. He can choose the least c 
structure shich «ill satisfy project require 
Furthermore, after a project has been coded and is d 
testing, he can make realistic assessments ccncernin 
effort which will be needed for program testi 

considering factors such as 

1. expected complexity of the project 

2. choice cf the programming techniques used 

3. organixatioE and experience of the programming 

4. available manp^ower and computer time for t 
pur poses. 

B. FDTDBE WCRK 

The analysis done on the NIDS programs and the r 
obtained fcr the measurement of program comp 

represents a modest contribution to the field of so 
engineering. But being far frcm complete or exhaust! 
following steps should be taken in order to 

additional validation of the analysis process. 
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1 . lurth€r €va luation of NTDS -M odules 

Additional NIDS modules should be evaluated in order 
to obtain larger sample sizes. It is realized that the 
evaluation process for the important modules is verj time 
consuming. However, the more important modules are used 
more freguentlj and will, in most cases, have a longer error 
history, which will provide valuable data for comparison 
with simulation results. 

2* Hvalua ti cn of stru ctured pr o gr ams 

It would be of interest in this respect to compare 
the evaluation of the NTDS-prccedures with procedures that 
perform the same functions but are rewritten and converted 
into a structured programmed form. It is expected that the 
structured programs would perform better with respect to 
error detection. 
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VIII. S UMMA BY AND CONCIUSICNS 



A metiiod to djEfine and analyze prcgran structures has 
keen presented. All measureinents obtained were based cn the 
descripticn cf the program structure in the fcris cf a 
directed crajh and the use of the error detection simulation 
mcdel. Ihis methcd has been used to analyze the procedures 
frcm two KIES modules. It was beyond the scope cf this 
effort tc obtain ccmparative results between this experiment 
and the actual error history cf the programs. However, it 
was possible tc obtain an initial guantitative assessmenr of 
measures cf complexity. 

By using this method to check program structures in the 
design phases it should be possible to produce programs with 
structures that are less complex and therefore easier and 
more economical to test and maintain. Also the methcd cculd 
he used during the test phase as a means cf assigning test 
resources. 
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8Ce CCMINUE 



APEENDIX B 



IIST CF EVALUATED PROGRAil STRUCTURES 



This list gives all the statistical data gathered frcic 
the ccnversicn of the procedures of the NTES modules into 
the fori of directed graphs. The abbreviations read as 
fellows : 



PUR 

R 

A 

P 

L 

Ss 

Mi 



Procedure 
Nuiber cf 
Nuttber of 
Nunber of 
Nuiber cf 
Number of 
Number of 



number within the module 

nodes (including transient nodes) 

arcs (including transient arcs) 

paths 

loops 

source statements 
machine instructions 



SA Sccrce stmts. /arc 

MA Machine instr./arc 
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1 



KcduJe One 



ENE 


a 


A 


P 


1 


2 


1 


1 


2 


14 


23 


22 


3 


4 


3 


2 


4 


3 


2 


1 


5 


4 


4 


2 


6 


34 


45 


64 


7 


4 


4 


2 


8 


13 


14 


3 


9 


4 


4 


2 


10 


5 


5 


2 


1 1 


6 


8 


5 


12 


2 


1 


1 


13 


2 


1 


1 


14 


6 


7 


4 


15 


4 


4 


2 


16 


14 


13 


1 


17 


14 


13 


1 


18 


21 


23 


2 


19 


19 


26 


7 


20 


45 


66 


1 1 


21 


35 


49 


88 


22 


25 


30 


1 1 


23 


7 


7 


2 


24 


6 


5 


1 


25 


12 


12 


2 


2 6 


6 


5 


1 


27 


8 


8 


2 


■2£- 


17 


19 


4 



SS 


MI 


S/A 


a/A 


2 


1 


2 . 00 


1. 00 


37 


134 


1.61 


5. 83 


5 


15 


1 . 67 


5.00 


18 


18 


9.00 


9.00 


4 


17 


1 .00 


4.25 


60 


302 


1 .33 


6.71 


8 


17 


2.00 


4.25 


10 


25 


0.71 


1. 79 


7 


15 


1.75 


3. 75 


4 


23 


0.80 


4. 60 


8 


15 


1.00 


1. 88 


4 


5 


4 . 00 


5.00 


5 


6 


5.00 


6.00 


9 


24 


I. 29 


3. 43 


6 


25 


1 . 50 


6.25 


13 


22 


1.00 


1.69 


13 


23 


1 .00 


1.77 


21 


35 


0.91 


1. 52 


22 


45 


1 . 16 


2. 37 


82 


134 


1 . 24 


2.03 


33 


ICO 


0.67 


2.04 


30 


53 


1 . 00 


1.77 


6 


8 


0-86 


1. 14 


8 


17 


1 .60 


3.40 


8 


26 


0.67 


2, 17 


5 


6 


1 .00 


1.20 


10 


20 


1 . 25 


2.50 


32 


99 


1 . 63 


5.21 



L 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

1 

0 
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I NH 

29 

30 

3 1 

32 

33 

5^1 

35 

36 

37 

38 

39 

40 

4 1 

42 

43 

44 

45 

46 

47 

48 

49 

50 

51 

52 

53 

54 

5 5 

56 



N 


A 


P 


L 


28 


32 


5 


0 


7 


10 


5 


0 


4 


4 


2 


0 


4 


4 


2 


1 


10 


9 


1 


0 


16 


17 


3 


0 


14 


17 


3 


0 


21 


26 


3 


0 


54 


64 


13 


0 


8 


10 


8 


3 


17 


25 


1 0 


0 


83 


120 


3704 


10 


33 


38 


7 


0 


12 


14 


2 


0 


13 


14 


83 


0 


27 


30 


7 


0 


12 


12 


2 


0 


9 


9 


2 


0 


19 


20 


4 


0 


23 


26 


7 


0 


15 


18 


7 


0 


2 


1 


1 


0 


9 


9 


2 


0 


125 


150 


1645 


o' 


1 1 


18 


9 


0 


34 


45 


5 


0 


6 


5 


1 


0 


46 


58 


13 


0 



ss 


KI 


S/A 


a/ A 


47 


150 


1.47 


4.69 


10 


43 


1 .00 


4.30 


4 


12 


1 .00 


3.00 


6 


13 


1.50 


3.25 


7 


7 


0.78 


0.78 


15 


23 


0.88 


1.35 


14 


20 


0.82 


1.18 


31 


57 


1 . 19 


2 . 19 


56 


111 


3.88 


1.73 


19 


40 


1 .90 


4.00 


17 


59 


0 . 68 


2.36 


78 


271 


0 . 65 


2,26 


31 


63 


0.82 


1.66 


1 1 


17 


0.79 


1.21 


8 


12 


0. 57 


0.86 


21 


38 


0.70 


1.27 


8 


13 


0 . 67 


1.08 


10 


25 


1.11 


2.78 


12 


22 


0.60 


1.10 


13 


34 


0.50 


1.31 


1 9 


47 


1.06 


2.61 


7 


38 


7.00 


38.0 


1 1 


36 


1 . 22 


4. 00 


102 


260 


0.68 


1.73 


1 1 


33 


0,61 


1,33 


63 


85 


1 . 40 


1.39 


7 


11 


1 . 40 


1. 89 


51 


86 


0.88 


1.48 
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£N6 

57 

58 

59 

63 

61 

62 

63 

64 

65 

66 

67 

68 

69 

7C 

7 1 

72 

73 

7 4 

75 

76 

77 

78 

79 

8C 

8 1 

82 

83 

84 



M 


A 


P 


L 


ss 


MI 


S/A 


a/ A 


30 


36 


14 


0 


26 


59 


0,72 


1.64 


40 


60 


216 


0 


49 


117 


0, 82 


1.95 


1 1 


12 


3 


0 


9 


15 


0-75 


1.25 


28 


37 


16 


0 


24 


52 


0.65 


1.41 


2 


1 


1 


0 


3 


5 


3,00 


5.00 


43 


62 


24 


1 


50 


96 


0.81 


1.55 


89 


140 


451 


7 


95 


214 


0.68 


1.53 


4 


4 


2 


0 


7 


14 


1 . 75 


3.50 


47 


56 


773 


3 


44 


129 


0.79 


2.30 


12 


12 


2 


1 


10 


16 


0.67 


1.33 


26 


27 


1 


0 


25 


44 


0.93 


1.63 


8 


8 


2 


1 


8 


30 


1 . 00 


3.75 


49 


61 


12 


0 


53 


90 


0.87 


1.48 


6 


7 


4 


0 


8 


22 


1 , 14 


3.14 


6 


7 


4 


1 


9 


23 


1 . 29 


3.29 


7 


7 


2 


0 


7 


16 


1 .00 


2.29 


8 


8 


2 


0 


7 


13 


0.88 


1.63 




6 


3 


0 


9 


19 


1 .50 


3. 17 


24 


28 


8 


0 


20 


47 


0.7 1 


1.68 


15 


19 


8 


0 


20 


45 


1 .05 


2.37 


17 


20 


9 


0 


1 0 


37 


0.50 


1.85 


10 


9 


1 


0 


5 


7 


0.56 


0.78 


25 


31 


3 


0 


23 


30 


0.74 


0.97 


44 


57 


1 1 


0 


55 


127 


3.96 


2.23 


8 


9 


3 


0 


7 


16 


0.78 


1.78 


6 


5 


1 


0 


8 


18 


1 . 60 


3. 60 


13 


14 


3 


0 


8 


20 


0.57 


1.43 


91 


120 


25 


0 


93 


191 


0.78 


1.59 
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NAP 



65 


33 


43 


219 


66 


18 


23 


13 


67 


2 1 


22 


6 


89 


51 


65 


14 


SO 


7 


10 


5 


91 


22 


30 


9 


S2 


5 


6 


3 


S3 


25 


34 


12 


S4 


7 


10 


5 


S5 


18 


27 


10 


S6 


45 


52 


35 


S7 


136 


211 


3972 



SS 


MI 


S/A 


M/A 


32 


93 


0.74 


2.16 


22 


56 


0.96 


2. 43 


25 


81 


0.93 


1.37 


54 


107 


0.83 


1.65 


8 


22 


0 .80 


2.20 


14 


47 


0. 47 


1. 57 


9 


28 


1 . 50 


4.67 


34 


132 


1.00 


3.88 


12 


37 


1 , 20 


3.70 


19 


58 


0.70 


2. 15 


40 


93 


0.77 


1. 79 


99 


162 


0. 47 


0.77 



L 

0 

0 

0 

0 

0 

0 

0 

0 

2 

0 

1 

0 
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2. 

ENE 


ecdul€ 

M 


T^o 

A 


P 


3 


2 


1 


1 


5 


6 


5 


1 


7 


2 


1 


1 


8 


2 


1 


1 


15 


1 1 


13 


5 


23 


1C 


1 1 


3 


40 


22 


27 


5 


4 1 


10 


14 


12 


46 


25 


37 


36 


47 


24 


34 


36 


48 


16 


21 


14 


54 


6 


5 


1 


c c 


8 


8 


2 


59 


6 


5 


1 


65 


6 


5 


1 


69 


4 


4 


2 


73 


18 


22 


10 


79 


13 


14 


5 


82 


23 


24 


2 


86 


30 


34 


6 


90 


13 


18 


8 


99 


25 


30 


10 


115 


6 


5 


1 


114 


4 


4 


2 


121 


6 


5 


1 


122 


18 


21 


6 


125 


37 


46 


13 


129 


9 


9 


2 


137 


13 


17 


1 1 


149 


18 


25 


9 



SS 


MI 


S/A 


M/A 


7 


7 


7.00 


7. 00 


3 


4 


0.60 


0.80 


3 


4 


3 .00 


4. 00 


9 


23 


9.00 


23.0 


12 


31 


0.92 


2.38 


12 


33 


1 . 09 


3.00 


14 


30 


0.52 


1.11 


13 


42 


0o93 


3.00 


34 


95 


0.92 


2.57 


34 


85 


1 . 00 


2.50 


17 


55 


0.81 


2.62 


10 


15 


2.0C 


3.00 


5 


13 


0.63 


1 . 63 


5 


10 


1.00 


2.00 


4 


12 


0.80 


2.40 


7 


15 


1 . 75 


3.75 


34 


97 


1 . 55 


4.41 


1 1 


34 


0,79 


2.43 


35 


64 


1 . 46 


2. 67 


33 


86 


0.97 


2. 53 


23 


71 


1. 28 


3.94 


23 


50 


0.77 


1.67 


5 


7 


1 .00 


1. 40 


3 


6 


0.75 


1.50 


7 


12 


1 . 40 


2.40 


20 


38 


0.95 


1.81 


37 


94 


0.80 


2. 04 


9 


21 


1.00 


2.33 


323 


53 


1 . 55 


3. 12 


35 


88 


1 . uo 


3.52 



L 

0 

0 

0 

0 

1 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

2 

0 

2 

2 

1 

0 

0 

0 

0 

2 

0 

0 

4 
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APPENDIX C 



DIBECTED GBAPHS 



CD the fcllcwing fages the structures of all the 
procedures are listed that were used as input data for the 
Error Detection Simulation Model. In addition tc the 
complexity measures used also listed are the results 
ohtained frcm the simulation, the average number of errors 
found with 1 input, ICO replications and 100 repetitions, 
and the percentage of expected errors detected. 

Differently tc the sample structure shown in Fig. 
2, the number cf statements is indicated in the following 
graphs only fcr arcs with ncnzerc instructicns. 



Ihe count for the number of nodes and the number of 
arcs includes the transient nodes (designated by letters) 
and the transient arcs (dashed lines) because they must be 
included into the inpufs for the Error Detection Simulation 
Program . 
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aodule: 1 



Procedure No.: 2 




Nuffter of nodes: 


14 


Numter of arcs: 


23 


Nuiiter of paths: 


26 


Nun her of sctrce stmts.: 


37 


Average errcr found: 


0.3144 


Percentage ericrs found: 


17.84 
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module; 1 



Procedure No. ; 8 




Nuiifcer of nodes: 


13 


Nuoiter of arcs: 


14 


Nuater of paths: 


3 


Nuirbet of sccice stmts.: 


10 


Average error found: 


0.2523 


Percentage errors found: 


52.98 
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Module 



1 



Procedure No . : 11 




Nuiiher of nodes: 6 

Nutther of arcs: 8 

Nuiiter of patis: 5 

Nuiiiet cf sctrce stmts.: 8 

Average error found: 0.1974 

Percentage errors found; 51.82 
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Med ule 



1 



Procedure No.: 14 




NuEter of nodes: 

Nuttfcer of arcs: 

Nueber of paths: 

Nuttter cf source stmts.: 
Average error found: 
Percentage errors found: 



6 

7 

4 

9 

0.2536 

60.34 



57 




Module: 1 



Procedure No. : 19 




Nuater of nodes: 


19 


Nuniter of arcs: 


26 


Nuaber of paths: 


7 


Number of source stmts.: 


45 


Average error found: 


0.2885 


Percentage errors found: 


27.54 
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( 



( 



Module: 1 



Procedure No.: 



22 




Nuifcer of codes: 


25 


Hunter of arcs: 


30 


Nuaher of paths: 


1 1 


Hunter of sccrce stmts.: 


30 


Average error found: 


0.4105 


Percentage errors found: 


28.74 
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Mcdule: 1 



Procedure No. 




Nuater of nodes; 12 

Nuiiter of arcs; 12 

Nuater of paths; 2 

Nuater cf sctrce stmts.; 8 

Average error found; 0.2324 

Percentage errors found: 61.01 





( 

I 



Module: 1 Procedure No.: 28 



4/16 



Nuafcer of nodes: 


17 


Number of arcs: 


19 


Number of paths: 


4 


Number of socrce stmts.: 


32 


Average error found: 


0.6400 


Eercentace errors found: 


42.00 
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( 



Mcdule; 




NuDb€£ of nodes: 28 

Sumber of arcs: 32 

Nuiiber of paths: 9 

Nuaber of source stmts.: 47 

Average errcr found: 1.3946 

Eercentace errors found: 62.31 



29 
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( 



Module 



1 



Procedure No.: 30 




Nuafcer of nodes: 


7 


Nuflifcer of arcs: 


10 


Nuiiter of paths: 


5 


Nuafcer cf scrice stmts.: 


10 


Average error found: 


0. 1649 


Percentage errors found: 


34.63 
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Module 



1 



Procedure Ho. : 3^ 




Nuffker of nodes: 16 

Hunter of arcs; 17 

Hunter of patts: 3 

Hunter of source stmts.; 5 

Average error found; 0,5465 

Percentage errors found: 76.51 
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Module 



1 



Procedure No. : 



35 




Nuater of nodes; 


14 


Number of arcs: 


17 


Number of paths; 


3 


Number of source stmts.: 


14 


Average error found: 


0.3576 


Percentage errors found; 


53.64 
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I 

I 

( 



ilcdule: 1 



Procedure Mo. : 36 




Hunter of ncces: 21 

Hunter of arcs: 26 

Hunter of paths: 3 

Hunter of sccrce stmts.: 31 

Average error found: 0-5203 

Percentage errors found: 35.25 
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Module 



1 



Procedure No 



39 




1/1 



Nuater of 


ncdes: 


17 


Numter of 


a res : 


25 


Nunher of 


paths: 


10 


Number of 


serree stmts.: 


17 


Average error found: 


0-2637 


Percentage 


errors found: 


32.57 
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I 



Module 



1 



Procedure Ho.: 44 




Nuirfcer of nodes: 27 

Nuffiber of arcs; 30 

Nuiiter of paths: 7 

Nunber of source stmts.: 21 

Average error found: 0,3554 

Percentage errors found: 35.54 
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I 



Module; 1 



Procedure No 



47 




Nuaher of nodes: 19 

Number cf arcs; 20 

Number of pa ths: 4 

Number cf source stmts.: 12 

Average error found: 0.4231 

Percentage errors found: 74.04 
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Module; 1 



Procedure No . : 48 




Nuirter of nodes: 


23 


Nuirter of arcs: 


26 


Nuiiter of paths: 


7 


Number of source stmts.: 


13 


Average error found: 


0.3287 


Percentage errors found: 


53.10 
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( 



Mcdule: 1 



Procedure Ilo . : 49 




Nuirter of nodes; 


15 


Number of arcs; 


18 


Number of paths; 


7 


Number cf source stmts.; 


19 


Average error found: 


0.2217 


Percentage errors found: 


24.50 
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Module 



1 



Procedure No . : 



53 




Nunher of nodes: 


1 1 


Nurher of arcs: 


18 


Nuffter of paths: 


9 


NuiEfcer of source stmts.: 


1 1 


Average error found: 


0.1876 


Percentage errors found: 


35.81 
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I 






( 

( 



( 

( 



Kcdule: 1 



Procedure No. : 57 




Nunher of ncces: 30 

Nuaber of arcs: 36 

Nuater of paths: 14 

Number of sctrce stmts.: 26 

Average error found: 0,2910 

Percentage eircrs found: 23.50 
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( 




( 



Module: 



1 



Procedure No.: 60 




Nuater of 


nodes: 


28 


Number c£ 


arcs : 


37 


Nunfcer of 


paths: 


18 


Number cf 


sctrce stmts.: 


24 


Average errcr found: 


0.3336 


Percentage 


eircrs found: 


29.19 
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I 

( 




Procedure No, 



75 



Nuiter of arcs: 28 

Nunher of paths: 8 

Number cf source stmts.: 20 

Average errcr found: 0-5433 

Percentage errors found: 57.05 
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i 



Module: 1 



Procedure No 



76 




Nuiiber of nodes; 


15 


Number of arcs: 


19 


Number of paths: 


5 


Number of sccrce stmts.: 


20 


Average errcr found: 


0.3893 


Percentage errors found: 


40.88 
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I 

I 



Module: 1 



Procedure Ho . : 11 




Hunter of nodes: 


17 


Hunter of arcs: 


20 


Hunter of paths: 


9 


Humber of sctice stmts.: 


10 


Average error fouxud: 


0.2425 


Percentage errors found: 


50.93 



11 





I 

I 



Module 



1 



Procedure No. : 79 




Nuater of nodes; 


25 


Number of arcs: 


31 


Number of paths: 


3 


Number cf sccrce stmts.: 


23 


Average errcr found: 


0.3628 


Percentage eircrs found: 


33.13 
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I 

I 



i 



Hcdule: 1 



Procedure No. : 81 




Nuiiiter of nodes: 8 

Nutcber of arcs; 9 

Nuiiter of paths: 3 

Nuffher cf scvrce stmts.; 7 

Average error found: 0.1449 

Percentage errors found; 43.47 
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'I I 




Module: 1 



Procedure No.: 86 




Nuater of nodes: 


18 


Nuniter of arcs: 


23 


Nunter of paths: 


13 


Nuoifcer of source stmts.: 


22 


Average errcr found: 


0.3370 


Percentage errors found: 


32. 17 
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I 

( 



Module 



1 



Procedure No. ; 87 




Nuiter of ncces; 


21 




Number of arcs: 


22 




Number of patts: 


6 




Number cf scuice stmts.: 


25 




Average errcr found: 


0. 


5029 


Percentace eircrs found: 


42 


.24 



81 






I 



( 



Module: 1 



Procedure No.: 91 




Nuirter of nodes: 


22 


Numter of arcs: 


30 


Nuiiter of patis: 


9 


Nuiter of sccrce stmts.: 


14 


Average errci found: 


0. 1438 


Percentage errors found: 


21.57 
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( 



Module: 1 



Procedure No. : 



92 




Nuaber of ccces; 5 

Number of arcs: 6 

Number of paths: 3 

Nuaber of sctice stmts.: 9 

Average error found: 0.1837 

Percentage errors found: 42.86 
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i 



Module 



1 



Procedure No 



93 




Nuaher of nodes: 25 

Nuniher of arcs; 34 

Nuiiter of paths: 12 

Nujcter of source stmts.: 34 

Average error found: 0.3972 

Percentage errors found: 24.53 
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i 

I 



i 



( 



Module 



1 



Procedure No. : 



95 




Nuaher of nodes: 


18 


Hunter of arcs: 


27 


Nunher of paths: 


10 


Nuaiter of source stmts.: 


19 


Average error found: 


0. 1822 


Percentage errors found: 


20.14 



85 







I 



Hcdule: 2 



Procedure No . : 15 




Nuafcer of ncces: 


1 1 




Nuiiter of arcs: 


13 




Nuirber of paths: 


5 




Nuttfcer of sccrce stmts.: 


12 




Average error found: 


0. 


0836 


Percentage errors found: 


35 


.55 
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I 

1 



( 



acdule; 2 



Procedure No 



23 




Nuiiter of ncces: 


10 




Nuttber of arcs; 


1 1 




Nuiber of paths: 


3 




Nuiber cf sccrce stmts.; 


12 




Average errci found: 


0. 


1592 


Percentage errors found: 


67 


.66 
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( 

( 



( 



Bed ule 



Procedure No.: 40 




Nuafcer of ncces: 


22 


Nuttber of arcs: 


27 


Nuater of paths: 


5 


Nuiiher of sccice stmts.: 


14 


Average errer found: 


0, 2018 


Percentage errors found: 


73.51 



88 




i 



( 

I 

\ 

( 

( 

( 

( 

( 



I 

I 

I 



( 



Med ule 



Procedure No 



4 1 




Nuaber of nodes: 


10 




Nuiiber of arcs: 


14 




Nunber of paths: 


12 




Nuaber cf setree stmts.: 


13 




Average error found: 


0. 


1554 


Percentage errors found: 


60 


.96 
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I 
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